Automatic head motion prediction from speech data
نویسندگان
چکیده
In this paper we present a novel approach to generate a sequence of head motion units given some speech. The modelling approach is based on the notion that head motion can be divided into a number of short homogeneous units that can be modelled individually. The system is based on Hidden Markov Models (HMM), which are trained on motion units and act as a sequence generator. They can be evaluated by an accuracy measure. A database of motion capture data was collected and manually annotated for head motion and is used to train the models. It was found that the model is good at distinguishing high activity regions from regions with less activity with accuracies around 75 percent. Furthermore the model is able to distinguish different head motion patterns based on speech features somewhat reliably, with accuracies reaching almost 70 percent.
منابع مشابه
Automatic speech/non-speech classification using gestures in dialogue
This paper presents an experiment carried out to determine what aspects of motion are associated with speech and what aspects are associated with non-speech in spontaneous dyadic communication. Six dialogs were analysed, and results show that the successful prediction of speech activity from motion differs considerably depending on the characteristics of the dialogue. The classification accurac...
متن کاملSpeech-driven head motion synthesis using neural networks
This paper presents a neural network approach for speech-driven head motion synthesis, which can automatically predict a speaker’s head movement from his/her speech. Specifically, we realize speech-to-head-motion mapping by learning a multi-layer perceptron from audio-visual broadcast news data. First, we show that a generatively pre-trained neural network significantly outperforms a randomly i...
متن کاملAnalysis of head motions and speech in spoken dialogue
With the aim of automatically generating head motions from speech, analyses are conducted for verifying the relations between head motions and linguistic and paralinguistic information carried by speech. Analyses are conducted on motion captured data during natural dialogue. Analysis results showed that nods frequently occur during speech utterances, not only for expressing dialog acts such as ...
متن کاملAutomatic Recognition of Eye Blinking in Spontaneously Occurring Behavior
Previous research in automatic facial expression recognition has been limited to recognition of gross expression categories (e.g., joy or anger) in posed facial behavior under well-controlled conditions (e.g., frontal pose and minimal out-of-plane head motion). We have developed a system that detects a discrete and important facial action (e.g., eye blinking) in spontaneously occurring facial b...
متن کاملPredicting Head Pose from Speech with a Conditional Variational Autoencoder
Natural movement plays a significant role in realistic speech animation. Numerous studies have demonstrated the contribution visual cues make to the degree we, as human observers, find an animation acceptable. Rigid head motion is one visual mode that universally cooccurs with speech, and so it is a reasonable strategy to seek a transformation from the speech mode to predict the head pose. Seve...
متن کامل